Natriello further suggests that the effects of these external and alternative
sources of education may serve as important control variables in evaluating
school effectiveness,  similar to parent educational levels and economic
resources.  The following items, he suggests, might be included on NCES surveys
to parents and students:

o private lessons in music and/or art

o private instruction in sports and other physical activities

(e.g., tennis, horseback riding)
o participation in a computer users group
o training related to a part-time job
o attendance at an ACT test preparation course

o remedial or supplementary instruction in one or more school courses
o training provided by a youth or community group, such as the YMCA or

the Boy Scouts or Girl Scouts

Data on participation in these non-school activities would help determine the
impact of participation on student performance and academic achievement.

Two other strategies for collecting this type of information are suggested by
Natriello. These are collecting information from educational services to the
community and collecting information from a representative sample of communities
and the non-school educational programs in those communities.  This strategy, he
says "would permit analyses to determine the distribution of supplementary
educational activities across communities with different demographic and
economic characteristics." The point of this sort of data collection is to
"begin to understand the extent to which non-school activities contribute to the
development of elementary and secondary students in the United States."

Finally, Murnane notes that more and more children are attending private schools
after their day at the public school has concluded.  The instruction they
receive at the private schools is intended to supplement their instruction in
the public schools. Murnane recommends that NCES learn more about after-hours
private schools in the United States. He suggests including a set of questions
on a new longitudinal study of American students that ask whether students
attend after-hours schools and, if they do, what the schools do and what they
cost. He recommends that the Census include a similar set of questions on the
October CPS survey.

RECOMMENDATIONS REGARDING ANALYSIS AND INTERPRETATION

Nearly three decades ago, the United States received a serious blow to its
self-esteem when the Russians launched Sputnik, becoming the first nation to
launch an artificial satellite. The reaction included sharp focus on our
schools, particularly the weaknesses in science and math.  A period of reform
followed, but before long we slipped back into complacency.

A Nation at Risk, the report of the Commission on Excellence in Education, has
once again dealt a blow to the Nation. We are in another period of reform. We
need to examine what is happening in the schools, what effects the various
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